Predictability in the large: an extension of the concept of Lyapunov exponent 
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We investigate the predictability problem in dynamical systems with many degrees of freedom 
and a wide spectrum of temporal scales. In particular, we study the case of 3D turbulence at high 
Reynolds numbers by introducing a finite-size Lyapunov exponent which measures the growth rate of 
finite-size perturbations. For sufficiently small perturbations this quantity coincides with the usual 
' Lyapunov exponent. When the perturbation is still small compared to large-scale fluctuations, but 

\q , large compared to fluctuations at the smallest dynamically active scales, the finite-size Lyapunov 

f"^ ' exponent is inversely proportional to the square of the perturbation size. Our results are supported 

\Q | by numerical experiments on shell models. We find that intermittency corrections do not change 

0^ ■ the scaling law of predictability. We also discuss the relation between finite-size Lyapunov exponent 

""^3 ' and information entropy. 
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PACS numbers 47.27.Gs, 05.45. +b, 47.27.Jv 



I. INTRODUCTION 



The ability to predict has been the single most important qualifier of what constitutes scientific knowledge, all since 
the successes of Babylonian and Greek astronomy. Indeed, the famous statement of Laplace that an intelligent being 
with complete knowledge of the present and of the laws of Nature will know the future for all time, assumes that 
the future is completely predicated by the past, and that perfect prediction would in principle be possible. In more 
mathematical terms one can say that in the physical sciences, whether in the classical or the quantum regime, one 
believes that Nature is ultimately described by differential equations, and if one knows them and how to solve them, 
one knows all there is to know about the world j[) . 

Laplacian determinism is always conditioned by the fact that in the real world initial conditions can never be 
known to arbitrary accuracy. More recent is the general appreciation of the fact that in the presence of deterministic 
chaos, predictability is even more severely limited, because small errors typically grow exponentially in time g]. Most 
sufficiently complex systems in the world display chaos. Therefore, most sufficiently complex systems can only be 
predicted for a finite time. However, there may be some aspects of a system that are stable, while others vary. To 
take a familiar example, weather prediction is possible, typically for about ten days on temperate latitudes, but how 
the wind blows on the corner of the street is in practice unpredictable from one moment to the next j^] . 

The words predictability and prediction are rather empty by themselves: one has to ask for predictability of what 
feature against what perturbation, in particular, against a perturbation of what size. In complex and spatially 
extended systems one can typically talk about large-scale features and small-scale features. The predictability of 
a small-scale feature against a small-scale perturbation is typically shorter than the predictability of a large-scale 
feature against a large-scale perturbation. Certainly this is not always true, and we will consider a counter example, 
2D turbulence, below. But it seems to be a very common situation. That successful large-scale prediction essentially 
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can mean just assuming that things do not change, is not per se an argument against prediction in the large. Also in 
weather prediction the assumption that the weather tomorrow will be like today is a fairly good one. 

In this work, a further development of results presented in the a recent brief report (J] , we will introduce a quantity 
which measures predictability in the large, and apply it to hydrodynamic turbulence. Before we proceed to the 
definitions, let us first recall some facts about predictability in the small, i.e. the effects of dynamical chaos. A system 
is said to be chaotic if small - i.e. infinitesimal - perturbations grow exponentially in time. If the initial perturbation 
is of size 5, and the accepted error tolerance is A, still small, then a rough estimate gives that the predictability time 
is 



where A max is the leading Lyapunov exponent |J]. 

Already within the framework of infinitesimal perturbations there are important modifications to (|l]) ^,0. In fact, 
in typical chaotic systems, (|l|) is not quite true 0. The exponent A max is a global quantity which measures the 
average exponential rate of separation of nearby trajectories, and fluctuations of the local exponential grow should be 
taken into account 0, but these effects are not what concerns us here. In this paper we shall address the problem of 
predictability in systems with many characteristic times, e.g. the case of fully developed turbulence where a hierarchy 
of different eddy-turnover times do exist, or when the threshold 5 is not small. In these cases the predictability time T p 
is determined by the details of the nonlinear mechanism responsible for the growth of the error In particular, T p 
may have no relation with the maximum Lyapunov exponent governed by the linearized equations for the infinitesimal 
error. In general, in this case the predictability time strongly depends on the details of the system 

According to Oseledec theorem, the leading eigenvalue of the linearized equations of motion is exp(A max t), except 
on a set of points of measure zero. The sub-leading eigenvalues have the form e Xit , and, taken together, the leading 
and sub-leading Lyapunov exponents measure the growth rate of d-dimensional volumes spanned by d infinitesimal 
vectors, where d can range from one to the dimensionality of the space where the motion takes place. 

In dynamical systems, in addition to Lyapunov exponents, an important dynamical characterization is given by 
the Kolmogorov-Sinai entropy, which measures the bandwidth necessary to observe a system over time, so that it 
could later be faithfully reproduced from the observations JBj. Arguing heuristically, new observations are necessary 
if an error grows in time, and the necessary rate of accumulation of information is the growth rate of the error, and 
therefore one expects Pesin theorem, 

^KS = A " ( 2 ) 

A;>0 

to hold true for a large class of systems. 

It was realized by Shannon that with finite error tolerance, the relevant quantity is the bandwidth necessary to 
observe a system such that it could later by reproduced within this error, not to arbitrarily high accuracy. This qua ntity 
was called by "source entropy with respect to a fidelity criterion" by Shannon , and "e-entropy" by Kolmogorov ||Tlf , 
who found analytic formulae, valid for Gaussian variables and Gaussian stationary processes. Recently the concept 
of e-entropy was taken up by Gaspard and Wang, who computed it from experimental data in thermal turbulence 



|12j, and for a very large variety of model problems in stochastic processes and statistical physics 13 1. We especially 
recommend their lucid and remarkably complete review ]l3| . 

However, the e-entropy does not say all one wants to know about predictability in the large. Just as the Lyapunov 
exponent is often more relevant, and more easily computable, than the Kolmogorov-Sinai entropy, so the predictability 
time with respect to a finite perturbation should be determined by a quantity analogous to the Lyapunov exponent, 
and not by the e-entropy. 

The natural starting point in looking for such a quantity is the time it takes for a perturbation to grow from an initial 
size 8 to a tolerance A. We call this the (S, A) predictability time and denote it by T(<5, A). Generally speaking, the 
predictability time will fluctuate. The natural definition of the finite-size Lyapunov exponent is therefore an average 
of some function of the predictability time, such that if both S and A are in the infinitesimal range, we will recover 
the usual Lyapunov exponent, and an obvious choice is then 

In appendix [A] we discuss other possible definitions for the finite-size Lyapunov exponent and the relation with the 
e-entropy. 
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In contrast to infinitesimal perturbations, for finite perturbations the threshold A is typically not to be taken much 
larger than the perturbation S. What is interesting, and what makes finite-size Lyapunov exponents different from 
Lyapunov exponents for infinitesimal perturbations, is the dependence on 5. 

This paper is organized as follows: in section || we recall the multifractal approach to turbulence, and Lorenz 
approach to the predictability problem within the Kolmogorov theory. We show that there are no multifractal correc- 
tions to the results of Lorenz, but that the scaling range for the finite-size Lyapunov exponents is shorter. In section 
III we describe numerical experiments on predictability in shell models for 31? and 2D fully developed turbulence. In 
section |[y| we present the results from the eddy damped quasi normal Markovian (EDQNM) approximation for the 
shell model, and compare them with the results of section III. In section [y] we summarize our results and present 
conclusions. In appendix ^ we discuss alternative ways of defining the finite-size Lyapunov exponent. Appendix [b| 
contains a derivation of the Kolmogorov results of e-entropy for Gaussian processes and Gaussian random fields. In 
appendix ^ we apply the results of appendix |^ to space-time Gaussian fields with spectra as in Kolmogorov 1941 
theory of 3D turbulence, and to fictitious O-dimensional fields that describe shell models. We show that the finite-size 
Lyapunov exponent is the the same for any dimensionality, but the e-entropy largely depends on the dimensionality- 
dependent density of degrees of freedom. The results of sections || and [II were also presented in less generality in 



II. MULTIFRACTALS AND MULTISCALING 



Our understanding of high Reynolds-number turbulence is still mainly based on the fundamental contribution of 
Kolmogorov in 1941 |Q. We will here just discuss on a phenomenological level the predictions of the Kolmogorov 
theory and of its multifractal generalizations [p~5|— ^~7|| . 

Turbulence is a statistically stationary state of matter on macroscopic scales maintained by external forces. One 
considers only effects that are captured in a hydrodynamic level of description, that is the time evolution is supposed 
to be completely described by the macroscopic Navier-Stokes equations ]lS| ] 

d t v + (v ■ V)v — —VP + vV 2 v, (4) 
V • v = 0. (5) 

From the typical length scale L, the typical fluctuations of velocity on that scale V, and the viscosity v, we can form 
the Reynolds number, 

LV 

Re=^-, (6) 

which characterizes the flow. 

The multifractal model [p~5|— |l7|,p| consists in assuming that at scales much less than L, however sufficiently large 
that the action of viscosity is weak, the velocity differences assume a scaling form 



\u(x + l)-u(x)\= Ul ~V[ - ) . (7) 



Different values of h are assumed to occur according to a probability distribution, which also takes a scaling form 

dh. (8) 



Prob U E 



V 



L J ' V L 



£ x 3-D(h) 



L 



The function D(h) is the fractal dimension of the subset with scaling exponent h. The moments of the velocity 
differences on length scale I can be computed as 

I \ qh / I x 3_ - D (' i ) 



(\u(x + l)-u(x)\o}~V J ^-j 1-1 dh, (9) 

and, for small-Z (i.e. in the inertial range), the integral in (Q) can be evaluated by saddle point method: 



\u(x + l)-u(x)\*)~V*\j.\ , (10) 
C q =mm[qh + 3- D(h)]. (11) 
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The model is physically reasonable for a large set of possible choices of the function D(h), but not entirely arbitrary. 
By normalization, the value of D(h) must always be less or equal than three, and the maximum must be obtained for 
some h. That is 

3-D(ft)>0. (12) 

The function D(h) can have support only at positive h, because a negative value of h implies that velocity fluctuations 
in a local inertial frame of size I increase without limit as I tends to zero. The Navier-Stokes equations are derived 
under the assumption that all velocities are much smaller than the velocity of sound, and this condition would then 
no longer hold p7| . Furthermore, an exact result of Kolmogorov assures that £3 = 1, so that 

3h + 3-D(h)>l, (13) 



where equality is obtained for at least one value of h [|17| . The inequality (|13|) is the analogous for turbulence of the 
inequality f(a) < a for multifractal measures (§]• 

In terms of the multifractal model, the Kolmogorov theory is formulated by supposing that the function D(h) 
has support only at a single point. From (|l^) and ( |l3|) it then follows that this point must be h = 1/3, and that 
13(1/3) = 3. It further follows the Kolmogorov law 



Z V 

(\u(x + l)-u(x)n~V[-) . (14) 



P/3 

L J 

Energy dissipation per unit mass and time, e, has dimension V 3 /L. We could therefore also write the right-hand side 
in (14) in the more familiar form (e/) p / 3 

From (O) it follows by balancing in (^), that viscous forces become comparable with inertial forces at the Kol- 
mogorov scale r\ which marks the lower end of the inertial range: 

7] = LRe- 3/4 . (15) 

If there exists more than one value of h then each h selects a different damping scale r/(h). By using (Q) and balancing, 
one gets [ fL9| 

7]{h) = LRe- 1/(1+h) . (16) 

Lorenz investigated the predictability problem within the framework of the Kolmogorov theory pQ] . Let us assume 
that we have a disturbance on scale Z, and that it grows at a characteristic rate given by the turn-over time at this 
scale: 

I L / ^ 2/3 



^ = 7W)~vKi) ' (17) 

We can turn around ( |l^ ) and say that after a time t a disturbance will have grown large on all scales smaller than 

/Vt\ 3/2 

1(f) ~ L — . (18) 



The size of the disturbance will then be (uf^) 1 ^ 2 , since all the smaller scales contribute relatively little. If we call the 
difference between two fields S, we can rewrite ( |l7|) as the predictability time with respect to a perturbation of size 6: 



l /<r 2 



W~v{v)- (19) 

In other words, the predictability time of a perturbation of size 5 grows as S 2 in Lorenz scenario. The finite-size 
Lyapunov exponent thus decreases with the error threshold as 5~ 2 . Finally, we can insert 6 in (|l8|), and find how the 
error grows with time: 

S(t) ~ V^. (20) 
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The upshot of these simple estimates is that finite error growth and predictability in high Reynolds number turbulence 
are characterized by algebraic laws, very different form the exponential laws characteristic of infinitesimal perturbations 
in chaotic dynamical systems. 

We now turn to possible consequences of a spectrum of h's to the predictability problem, in direct analogy with 
the Lorenz theory. We start by assuming the inverse of (pf) that is 



Prob <^ — € 



h+dh 



dh, 



(21) 



where we have identified ui with S, and used (|^) to relate I and 5. The length I is now assumed a fluctuating quantity, 
and has the interpretation of a scale such that two fields are uncorrelated on all smaller scales, given that the distance 
of the two configurations is 5. 

The finite-size Lyapunov exponent is, according to (||), proportional to the expectation value of the inverse pre- 
dictability time: 



T(S) 



V 
L 



dh. 



In the small error limit we thus expect the finite-size Lyapunov exponent to scale as a power of the error size: 

2-D(h), 



(ns)) 



X 



minh 

h L 



(22) 

(23) 
(24) 



The exponent \ is always equal to the Lorenz value of —2. This follows easily from (p"3|), which can be rewritten in 
the form 



1 



D(h) 



> 



for all h, 



(25) 



where the equality holds for the exponent /13 which dominates the third order structure function. 

As far as we know this result is new. One could therefore conclude that the exponent x f° r the scaling of the 
finite-size Lyapunov exponent with error threshold is a new invariant of the multifractal approach to turbulence, and 
that the law (1/T(8)) ~ 5~ 2 can be easily observed in numerical experiments. This is not quite simple, due to the 
influence of the fluctuating cut-off ([l6]) . The smallest fluctuation in a field scaling with exponent h is 



(26) 



5(h) = VRe-~ h/{1+ ' h \ 

which inversely determines the smallest value of h contributing to a fluctuation of size 5. A modified version of (|2 
therefore reads 



(t(s)) 



V 
L 



l-l/h 



(3-D(h))/h 



dh. 



(27) 



The integral is dominated by /13 as long as 6 is much larger than 6* ~ VRe~ h3 ^ 1+,l3 \ For smaller 6 values, the 
integral is dominated by the lower end-point in (p7|), which leads to an intermediate dissipative range, in the sense of 
Frisch and Vergassola p5). As a consequence, we have 



(t(S)) 



■^rnax 


for 


5 < Stymax), 


§ x(S) 


for 


S(h max ) < 6 < S* 


s- 2 


for 


5>S* 



with 



X(S) = 1 + 



2-DQi) 



(28) 



(29) 



where h and 8 are related via (|26|). From (13) follows that /13 is less or equal to 1/3 in the multifractal approach, and 
therefore the bottom of the scaling range of the finite-size Lyapunov exponent is larger than the corresponding error 
size in the Kolmogorov theory. The scaling range for the finite-size Lyapunov exponent thus is generally shorter in a 
multifractal model. 
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III. ERROR GROWTH IN SHELL MODELS 



Simplified dynamical models of fluid turbulence with relatively few degrees of freedom, collectively referred to as 
shell models, have been studied since the seventies. These models by construction typically include a Richardson 
cascade of energy from large to small scales. Some of the models are dynamically stable, with a fixed point which 
reproduces the Kolmogorov law for the energy spectrum, E(k) ~ fc~ 5 / 3 . An historical overview, with references to 
much of the early work, can be found in a recent monograph |22| . 

More interestingly, other models are dynamically unstable, with chaotic motion taking place on a strange attractor 
where the Kolmogorov 5/3 law holds to good accuracy, but not exactly. One of the simplest example is the family of 
models introduced by Gledzer |^3| , and Yamada and Ohkitani Jp3, now commonly called the GOY models pa] . They 
have recently been the subject of several investigations |^,^,0^8|j2^] . An in-depth description of this work can also 
be found in p2| . 

The GOY models are defined as follows: Fourier space is divided into n = 1, . . . , N shells, labeled by the wave- vector 
modulus k n = fco 2™, where k$ is a constant. The velocity difference over a length scale l n ~ k~ l are represented, each 
by one complex variable u n , which obey the following system of coupled ordinary differential equations: 

^-u n = -vk\ + ig n + f SnA (30) 

(31) 

where / is the strength of the external force, acting on large scales, and v the viscosity. For any values of the three 
coefficients a, b and c, phase space volume in preserved in the force- free inviscid limit. 

The restricted number of degrees of freedom is both the main advantage and disadvantage of shell models. It 
is an advantage, because it allows simulations at much lower viscosity and for much longer time than in the full 
Navier-Stokes equations. But it is also a severe departure. All spatial structure of the field is ignored. 

One of the coefficients in (^lj), say a, can be scaled to one and the condition of energy conservation fixes one more, 
such that, in terms of one parameter e, b is equal to — e and c to — (1 — e). With e greater than one the GOY equations 
conserve one more positive definite quantity besides energy, i.e. an analogous situation to 2D hydrodynamics. The 
dynamical behavior of the GOY models in this range is rather far removed from 2D turbulence as was shown in recent 
papers [ (30"pl| ] . We include below a study of predictability in such shell models, but just as a simplified model to 
demonstrate one possible scaling behavior of the finite-size Lyapunov exponent. 

With parameter e in the range between and zero and one, the GOY models also preserve another invariant, but 
which is not positive definite. In the following we will look at the GOY model with the standard choice of e equal to 
1/2, i.e. 

o=l, & = --, c = - 1 -. (32) 

The second invariant then has physical dimension of helicity p8| . Presumably that is the reason why this particular 
model has turned out to be so close to numerical and experimental data on Navier-Stokes turbulence in 31? [ B2LB3 |. 

Before we turn to the numerical experiments, let us summarize some salient features of the system defined by 
(|30|)-(|32"|). Energy is pumped into the system by the force, which acts only on shells with low values of n, and is 
removed at high shells by viscosity. From k$, v and the typical fluctuations of velocity on large scales, V, we can 
form a Reynolds number Re = V/k$v. We consider the situation where Re is large, such that there is a wide range 
in n where the external and viscous forces are both negligible compared to the inertial forces. In this inertial range, 
we have (|u„| 2 ) ~ k~&, where the exponent £2 is close to 2/3 [|tJ. 

An estimate of the smallest excited scale is in analogy with the Kolmogorov scale k n * ~ koKe 3 ^ 4 . The slowest 
dynamical scale is the time-scale of the the shells containing most energy, about l/k V, and the fastest is t~} ~ 

kn* (l^n* I 2 ) 1 / 2 , or, about koVKe 1 ^ 2 . From the fastest time-scale follows by dimensional analysis in the Kolmogorov 

1/2 

theory that the leading Lyapunov exponent should grow with Reynolds number as Re ' , a prediction due to Ruelle 
p4| . In the multifractal picture there are corrections to this estimate and that the leading Lyapunov exponent of the 
GOY shell model scales as Re Q where 



max 

h 



D(h) -l-2h 



1 



(33) 



It is indeed numerically observed to scale as Rc ' 495 , in good agreement with a computation of a staring from a 
function D(h), obtained by a parametric fit of measured values of the scaling exponents C, q in experiments fll^fl- 
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The mean square fluctuations at the Kolmogorov scale are (\u n * | 2 ) ~ Re . If we compute the distance between 
two shell variable configurations as 



(34) 



an error smaller than 0(Re 1 ^ 4 ) is relatively small all over the inertial range. It can be taken to be infinitesimal, and 
its growth rate will be the fastest linear growth rate. 

If, however, the error is larger than (9(Re -1 / 4 ), it could be larger than the typical size of the fluctuations at the 
Kolmogorov scale. Such an error would have to be concentrated on larger scales, since otherwise we have that for some 
n u n and/or u' n is much larger than the typical size. In other words, a physical perturbation larger than O^Re^ 1 ^ 4 ) 
cannot be obtained by a random perturbation of that size uniformly distributed over all the shells. 

In the Appendix [a] we discuss some possible definitions for the finite Lyapunov-exponents. In what follows we adopt 
the following procedure: after a long integration time to let the system relax towards the statistically stationary state, 
we introduce a very small error. This is done by generating a new shell variable configuration u' n differing from u n 
by a small fraction of (Iwnl 2 ) 1 ^ 2 ■ Another possible approach would be, that if we want an initial error of size e, we 
determine a shell n e such that (litnj 2 ) 1 / 2 is about e, and we concentrate the perturbation on shells above n t . 

We then iterate u n and u' n (perturbed system) for again a long time, such that the error has grown to a threshold, 
which is still small compared to VReT 1 ^. We thus have two realizations of configurations in the statistically stationary 
state, which only differ by a small error, which we call 5$. 

Further we define a series of thresholds 5 n — r n 5o, and we measure the times it takes for the error to grow from So 
to Si, and so on. For brevity we will call these times error doubling times, even if r can be different from two. The 
threshold rate r should not be taken too large, because then the error has to grow through several different scales 
before reaching the next threshold. On the other hand, the rate r can not be too close to one, so a sensible threshold 
rate is on the order of two. The most convenient choice of r clearly depends on the how the fluctuations in the shell 
variable depend on n, and in what way we measure the error. For our model (|3l]) with error measured by (|34|), the 
range of error sizes in the inertial range is not large, scaling as Re -1 / 4 . For 35 shells, which is the largest systems we 
simulate, we can take Re equal to 10 10 , which gives an error range of about 300. With error threshold rate equal to 
two, that would give about 8 data points, with some points lost on both ends due to boundary effects. For practical 
reasons we therefore take r equal to \/2. 

When we have performed N error doubling experiments, we can form an estimate of the expectation value of some 
quantity A: 



1 N 



(=i 



This is not the same as taking a time-average, since different error doubling experiments may take different times. 
Indeed, we have 

(A) t = A(t) dt = ^___2 = <4l>_. (36 ) 

A particular case of the above relation concerns the mean error doubling times themselves. Let T r (5 n ) be the time it 
takes for an error to grow from threshold 5 n to <5„+i. Then 



T r {S n ) 1 1 (T r (S n )) f 

where we have used the definition of (|J). 

The finite-size Lyapunov exponents, A(<5„), can be compared with shell turn-over times as follows: we first select 
a shell ns such that (\u ns I 2 ) 1 / 2 is about S, and then estimate t^ 1 as k ns S, which scales as 8~ 2 . This argument for 
typical error growth times is the same as Lorenz argument for 3D turbulence Q , discussed above in section |TJ. 

In Fig.[l] we compare error doubling times and shell turn-over times, as a function of size of the perturbation and 
of the typical fluctuations in the corresponding shell. Below the Kolmogorov scale, the turn-over times increase: 
we are here in the dissipation range, where the shell amplitudes decrease quickly. On the other hand, the doubling 
times tend to a constant as the error threshold is small. We are here in the infinitesimal range, and the constant is 
approximatively the inverse of the Lyapunov exponent. At the Kolmogorov scale, there is rather large discrepancy 
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between the Lyapunov exponent and the turn-over time. This observation, that the Lyapunov exponent obeys a 
scaling law with a sizeable numerical pre-factor, has been made before [p7| ; but without a plausible explanation. We 
here find nice agreement with our prediction that the inertial range for the finite-size Lyapunov exponent is shorter 
than the spectral inertial range, because the first is limited from below by the scaling exponent /13, as in equation 
(p6|), while the second is limited from below by the scaling exponent h% [ pi") . 

In Fig. we compare the error doubling times for different Reynolds numbers. For small thresholds the doubling 

. — 1 /2 

times scale as the Lyapunov exponent, i.e. as Re ' . We also observe that the bend away from the infinitesimal 
growth rate occurs at smaller error scales for larger Reynolds numbers. This suggests that a simple scaling ansatz 
can be sought in the following form: times and errors are scaled with the turn-over time and the typical scale of 
fluctuations at the Kolmogorov scale, that is by Re -1 / 2 and Re" 1 / 4 , respectively. In Fig. || we show such re-scaled 
data. The data collapse is reasonable. 

To improve the data collapse, taking into account multifractal corrections, we made a scaling based on multiscaling 
plfl , i.e. of the form 

M./W^/W/h^W) (38, 

where R Q , V are parameter to be fixed, and f(x) is the scaling function. According the argument at the end of section 
||, we have f(x) ~ x~ 2 for large x, while f(x) is constant for small values of x. In the intermediate regime f(x) has 



a nontrivial form which depends on the shape of D(h), as follows from eqs. (pq)-(29). The result is shown in Fig. 
The data collapse is clearly improved. 

We conclude this section discussing the case of 2D turbulence. Two dimensional Euler equation has the peculiar 
property of an infinite number of invariants. Two of them are retained in a finite Fourier discretization, the energy 
and the average square vorticity, or enstrophy. As we previously discussed, the second conserved quantity in the GOY 
shell model depends on the choice of the parameter e. With the choice e = 5/4, leading to 

a = l, b = ~, c=i (39) 



equations (30) conserve in the unforced and inviscid limit, in addition to the energy, the enstrophy here defined as 

Z=lj2 k n\ U ^- ( 40 ) 

n 

Despite the fact that the two-dimensional shell model has superficially the same physical justification of its three- 
dimensional corresponding model, it has been demonstrated that it has little to do with turbulence (3^j3l[]. Moreover, 
all the numerical simulations of two-dimensional Navier-Stokes equation at sufficiently high Reynolds number have 
demonstrated the dynamical relevance of coherent structures which emerge spontaneously from the turbulent flow. 
The predictability problem, which is more relevant for geophysical flows in this case than in 3-D turbulence, is also 
ruled by coherent vortex motion in the physical space, rather than modes dynamics in Fourier space f^f . 

With this limitations, the study of the predictability problem, as addressed in the present paper, in two-dimensional 
shell model is nevertheless interesting because of the different scaling behavior with respect to the 3D situation. 
Dimensional analysis ]36| shows that in the enstrophy cascade one expects constant - i.e. independent on the scale - 
turn-over times. Hence an argument similar to that of section |l| shows that 

const = A max / In r, (41) 



T r (Sv) 



where A max is the largest Lyapunov exponent. The predictability time for two-dimensional shell model is thus 

— 1/3 

determined by a single value I/A max ~ e z , where ez is the enstrophy flux toward small scales, up to a perturbation 
of the order of the large scale velocity field, where saturation effects are dominant. Figure || shows the finite-size 
Lyapunov exponent X(Sv) for a simulation with N — 24 shells. The forcing term is now / = 5 x I0~ 4 (I + i) and 
v = I0~ 8 . Because of the inverse energy cascade we introduce in the equation ( |30|) an artificial large-scale dissipation 
u'/kn [y' = 10" 5 ) in the first shells n < 3 |§. 

The velocity field shows a pseudo-cascade power law, see [|o), (lunl 2 ) 1 / 2 ~ e^ 3 ^" 1 in a wide range I < k n < I0 3 . 
The mean enstrophy flux in this range is ez — 5 x I0~ 6 which is in agreement with the dimensional evaluation of the 
eddy turnover time t„ ~ e~J 3 . 

The data plotted in figure [| are obtained by using the method described in appendix [A| for computing the size 
dependent Lyapunov exponent. The same results, not reported, can be obtained from the doubling time algorithm. 
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We stress once more that, in the light of the results discussed in [|30|, two-dimensional shell model is not a good 
model for two-dimensional turbulence. As a consequence a discussion of the predictability problem for two-dimensional 
turbulence requires the direct study of Navier-Stokes equations |Q. Preliminary results show that this scenario 
remains, nevertheless, valid in the direct cascade fl37j. 



IV. CLOSURE APPROXIMATION 



In this Section we describe the results obtained from the eddy damped quasi-normal Markovian approximation 
(EDQNM) for the shell model. The basic idea of closure approximations is quite simple: write down the Reynolds 
hierarchy for moments of the shell variables and truncate the chain to the lowest sensible order. The important 
point is that in the closure approximation intermittent effects are washed out, so we can directly test if the relevant 
mechanism is due to the existence of many characteristic times. We do not report the derivation of the EDQNM 
equations for the shell model. The interested reader can find it in Ref. |38|. 

We consider two independent realizations of the shell model field, u n and v n , with the same energy spectrum 
E n = (u„u*) = (v n v n ), and both evolving according the shell model equations (f30|)-(|32|). The distance between the 
two fields can be obtained from, cfr. (Em|), the energy difference at shell n: 



A„ = - ((«„ - v n ) « - «*)) = (E n -W„), 
where W n = (u n u*), and 5i denotes the real part. . From the definition it follows 

1/2 



6v(t) = 



(42) 



(43) 



The evolution equations of E n and W n in the EDQNM approximation read: 



2vkZ) E„ 



k n {En+1 E n +2 — 2 ^ n ^n+2 — 2 111 " r -" 

-A k 2 



.^(n-l.i) (E n E n+1 
2, t) (E n -i E n 
+2e5 nA , 



-EE 

2 ^n 
2 En-1 E n +i — \ E n _x E n ) 
\ E n -2 E„ 



h E n _2 E n _i 



(44) 



and 



'A 

v dt 



2vkT] W n = 2 



kl 6(n, t) (W* +1 W* +2 - | W n E n+2 - \ Wn E n+ l) 

! 6{n - 1, t) {W n E n+1 - \ Wl_ x W* +1 - \ E„_! Wn) 
2,t) (E n ^W n -±E n _ 2 W, 

+2ed n A, 



-i k 2 

2 ^n- 



-W* nW* 

2 n— 2 " n-U 



(45) 



where 



and 



9(n,t) 



(46) 



fi n = fj,(k n , E n ) = a k n E n 



1/2 



(47) 



We have one free parameter, the dimensionless constant a. It should be adjusted such that the spectrum is as similar 
as possible to the spectrum obtained in simulations of the full equation. The energy spectrum of the shell model 
in the EDQNM approximation must therefore obey E n ~ C(a)e 2 ^ 3 k n 2 ^ 3 in the inertial range. The undetermined 
function C(a) is the Kolmogorov constant. 

On the other hand it has become clear in several independent investigations that intermittency corrections exist 
in shell models. The energy spectrum is therefore in reality more closely described by E n ~ F(€)k^ 2 1 where the 
exponent (2 has been estimated to be 0.70 ^?J. The function F that gives the prefactor to the power law in the 
inertial range should not depend on viscosity, but depends on the forcing through e, the mean dissipation of energy 







per unit time, or, equivalently, the mean energy input into the system from the force. In a really large inertial range 
the two power-laws are not good approximations to one other. The best that can be done is to demand that the 
spectra agree as closely as possible at the upper end of the inertial range. A reasonable agreement is obtained for 
a = 0.06, leading to C(a) — 1.5 which is the value observed both in simulations of the shell model and in experiments 

& 

The procedure described in the previous Section to compute the scale dependent Lyapunov exponent for the shell 
model can be adopted here for the closure equations. In practice after a long iteration time, to have a well stabilized 
energy spectrum E n , we take a small initial distance <5v(0) and perform the doubling experiment similar to those of 
the previous section iterating equations (ff5j). 

In Fig. ^ we show (l/T r (Sv)} /Re 1 ^ 2 as a function of the rescaled distance 5v/Re~ 14 , for different Re = v~ x . The 
other parameters are N = 32 shells, kg — 0.05, integration step 10 -6 and r — 2 1 / 2 . From Fig. ^| we see that the 
closure approximation leads to the same scenario observed for the shell model, confirming that this is due to the 
existence of many characteristic scales. We note that the slope of the curve for Sv/Ke^ 1 ^ 4 > 10 is the Lorenz value 
—2 since in the EDQNM approximation there are not intermittent corrections. 

We note that in this case, since there is not intermittency, the effective inertial range roughly coincides with the 
inertial range. In Fig. [?] we compare (l/T r (6v)) and the inverse of the turnover time r~ 1 (n) = k n En as a function 
of the distance Sv. In the figure we used r = 2. 

V. CONCLUSION 

We have introduced a generalization A(<5) of the leading Lyapunov exponent to finite perturbations of size S. Unlike 
the Lyapunov exponent, A(<5) contains direct information on the predictability time for an extended chaotic system 
and it is particularly useful in presence of several characteristic times. Moreover, it is computationally no more 
expensive than the standard Lyapunov exponent. 

In the limit of infinitesimal perturbation 6, the finite-size Lyapunov exponent gives the leading Lyapunov exponent. 
The way in which this limit is reached depends on the details of the particular system and gives informations about 
the characteristic time scales. In the case of 3D fully developed turbulence we have found an universal scaling law 
A(<5) ~ 5~ 2 where the value —2 of the exponent is an invariant of the multifractal approach. 

The scaling law is confirmed by extensive numerical simulations on shell model for turbulence at very high Reynolds 
numbers, but we warn that it would be very difficult to observe such a scaling in experimental data because of the 
reduction of the scaling range in presence of intermittency. 

We have also shown and discussed the relationship between our approach and the e-entropy results in literature. In 
conclusion, we think that it would be useful to compute the finite-scale Lyapunov exponent in other complex dynamical 
systems, as a new tool for investigating the presence of characteristic times and the predictability properties. 
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APPENDIX A: FINITE SIZE LYAPUNOV EXPONENTS AND e-ENTROPY 

Here we discuss some alternative ways of computing the finite-size Lyapunov exponent beyond the definition (|^). 

The first method is a modification of the standard technique f40|JlT| . We integrate two trajectories u(t) and u'(t) 
with initial Euclidean distance Su(0) — 6 until their separation becomes larger, at a given time T r than rS where r is 
a given constant coefficient. The perturbed trajectory u'(T r ) is then rescaled at the original distance 5, keeping the 
direction u' — u constant. The possible problem with this definition is that it assumes that the statistically stationary 
state of the system is homogeneous with respect to perturbations of finite size. One may plausibly argue that the 
structure of the attractor in phase space on which the motion takes place may be fractal, and not at all equally dense 
at all distances from a given point. The procedure outlined would then not necessarily sample in a faithful way the 
motion on the attractor. In practice, as it is showed by the numerical experiments, we do not find any difference 
with the numerical method presented in the main text, so the effect must be quite small. For the usual Lyapunov 
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exponent the problem does not exist since we only use small finite perturbations as an approximation to infinitesimal 
perturbations. 

The hnite-size Lyapunov exponent at scale 6 is obtained by averaging the divergence rate 

along the unperturbed trajectory u(t). The average (• • -) e is over many error-doubling experiments. 

In the case of infinitesimal error S, and not too large factor r, this definition leads to the maximal Lyapunov 
exponent A max . 

For maps, the above methods have to be slightly modified since the distance between two states is not continuous 
in time. From eq. (|37]) readily follows 

where T r is the minimum time such that the distance between two realizations is larger or equal to rS and 8{T r ) is 
the distance at that time. 



Definition (Al) is valid for any value of r, which however is generally assumed not too large to avoid the interference 
of different scales. In particular, one could think of removing the threshold condition used for defining T r and simply 
compute the average error growth rate at every time step. In other words, at every time step St in the integration, the 
perturbed trajectory u'(t) is rescaled to the original distance <5, keeping the direction u — u 1 constant. The finite-size 
Lyapunov exponent is given by the average of the one-step exponential divergence: 

w-iMm®)). (A3) 



St \ \ 5u{t) 

which is equivalent to the above definition (|AIj ). This method is indeed the one used for computing A (8) in the case 
of 2D shell model. The one-step method ( |A3[ ) has the advantage that it can be easily generalized to compute the 
sub- leading finite-size Lyapunov exponent following the standard orthonormalization method pcj . One introduces 
k perturbed trajectories . . . each at distance S from u and such that vP*> — u are orthogonal each to the 

others. At every time step, any difference — u is rescaled at the ori gina l value and orthogonalized, while the 



corresponding finite size Lyapunov exponent is accumulated according to (A3). Here we have again the problem of 
the implicitly assumed homogeneity of the attractor, but also a problem of isotropy when we re-orthogonalize the 
perturbations. We note that this could be a more serious problem, which will not be discussed here any further. 

For systems with only one positive Lyapunov exponent the size dependent Lyapunov exponent A(<5), definition (|A2|) , 
or (||) , coincides with the e-entropy widely described below in Appendix [B| and by Gaspard and Wang Jl^,[l3| . We 
consider now few examples. 



1. Map with noise 



Consider a chaotic deterministic map perturbed by a Gaussian term: 

x(n + 1) = f(x(ri)) + ay(ri), (A4) 

where y(n) is a stochastic variable with Gaussian distribution of zero mean and unit variance. The y(n) for different 
times n are independent. When a = the map f(x) is chaotic with positive Lyapunov exponent A. A simple 
computation shows that 

5x(n) ~ Sx(0)e Xn + aSy(n- 1). (A5) 

For [fa(0)l <C cr the noise term is negligible, so that A(|<5x(0)|) ~ A. In the opposite limit, |5x(0)| 3> cr, the first term 
in QA5D can be neglected so that Sx(n) ~ a Sy(n — 1). Thus i n on e iteration of the map the distance between the two 



trajectories grows to 0(a), larger than the tolerance. From (A2) we have A(|fe(0)|) ~ ln(cr/|fa(0)|). These are the 
same result obtained for the e-entropy, see Sect. 3.5 of Ref. |13|. 
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2. Ornstein-Uhlenbeck process and Yaglom noises 



Consider the Gaussian process described by the Langevin equation 



dx 

— = -ax + c-q, 



(A6) 



whe re a > and 77 is a white noise with zero mean and correlation (rj(t) ?7(t'))rj = S(t — t'). The formal solution of 
(|A6|) reads 



x (t) = e- at x{0) + c / e- a{t - t ' ) r){t')dt\ 
Jo 

so that the distance between two different process behaves in time as 

5x{i) = e- at 6x{0) + c f e - a ^-^ Sr,(t') dt' . 



(A7) 



(A8) 



This implies that 8x(t) ~ y/ci, and thus the predictability time T(Sx(0), A) behaves as T(5x(Q), A) ~ (A/c) 2 so that 
\(6x(0), A) ~ (c/A) 2 ~ (c/Sx(0)) 2 , as found for the e-entropy, see below or Sect. 3.6.2 of [[l3| . 
Similar computations can be done for the Yaglom noise, see Sect. 3.6.3 of |13|. 



3. Model of deterministic diffusion 

The one-dimensional map 

x(n + 1) = x(n) +psin(27rx(n)), (A9) 

presents deterministic diffusion. For example for p = 0.8 the diffusion coefficient is D ~ 0.18. The size dependent 
Lyapunov exponent for this map can be computed numerically using the definition ( |A2| ). From Fig. |^ we see that 
X(S) ~ A for small 6, while \(5) ~ 6~ 2 for large values of 5. The e-entropy shows the same behavior, see Fig. 25. b of 
Ref. Jig}] . Note that in Ref. (l3| the e-entropy is measured in unit of digit/iteration, so there is a multiplicative factor 
In 10 ~ 2.3 with X(S) of Fig.W 



APPENDIX B: THE KOLMOGOROV RESULT ON e-ENTROPIES OF GAUSSIAN PROCESSES 

The purpose of this appendix is to derive systematically on a physical level of rigor the results of Kolmogorov on the 
e-entropy of Gaussian variables and Gaussian processes and random fields. These results were stated without proof 
by Kolmogorov |llj , and in the review by Tikhomirov p2l . The published proofs we are aware of are either not easily 
accessible, or carried out in such generality that the simple underlying idea may be lost to the less mathematically 
inclined reader. Hence the interest of including a simple derivation here. Let us just for completeness refer to 
comparatively recent paper |Q . 

The general setting is that of two random variables £ and 77, taking values in spaces X and Y. The values in two 
realizations of £ and r] are denoted x and y, respectively, where x lies in X and y lies in Y . When X and Y are 
continuous the probability distributions of £ and 77 will be denoted P^{dx) and P v {dy). Except for the trivial case, 
the variables £ and 77 are not independent, but characterized by the joint probability distribution P^_ n (dx,dy). 

The single- variable probabilities are determined from the joint distribution as 

P i (dx)= [ P t>v {dx,dy) P v {dy)= [ P t>v {dx,dy) (Bl) 

J y J x 

The conditional probabilities are 

P ( ^(dx,dy) P^ v (dx,dy) 

P ^ {dx) = P n (dy) P ^ {dy) = P 6 (dx) (B2) 
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1. Preliminaries from information theory 



We suppose in this subsection that the spaces X and Y arc discrete, and consist of finitely many points, 
and yi , . . . , y m , and associated probabilities p\ , . . . , p n and qi , . . . , q. 
The entropy of the random variable £ is 
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H (0 = -X>lnpi (B3) 
i=l 

If we want to code a message of ./V letters, which are given by N consecutive independent realizations of £, so taking 
values in x±, . . . x n) this can be done, in the limit when N is large, by using H(£)/ In 2 bits per letter |L0[ . The pair 
£ and 77 specify both a source signal and the output after sending the signal through a channel of transmission. The 
joint probability distribution P^ jV can be decomposed either to the pair £ (an input) and (an output, given the 
input), or to the pair 77 and £\r]. In the discrete case these second conditional probabilities are denoted p^u, where i 
ranges from one to n, and j ranges from one to m. The equivocation is 



(H(m)y 



X! /y ' ( yii'- j ln /'< j • ( B4 ) 



Shannon []10[1 considered a gedanken experiment where we send an error-correcting message parallel to the transmission 
of £ to 77, and asked for the number of bits in the error-correcting message needed to transmit N letters in the original 
message virtually without error. In the limit of large N, the answer is (H(£ \rj)) y / ln2 bits per letter. 

Suppose further that we have some source of information which we can recode into letters from £, then transmit to 
77, and then observe. The rate of transmission of information is not the source entropy, since we must correct for the 
equivocation, but the mutual information: 

i&v) = m) - (m\v))y (B5) 



If we introduce the discrete joint probabilities pij and rearrange terms, we see that (B5) can be rewritten as in a 
more symmetrical way, namely: 

mr,) = Y J Pi,o^— (B6) 

A channel of transmission of information can be considered as a collection of pairs £ and 77, where the £'s are 
the possible inputs, and the 77's the corresponding outputs. The capacity of the channel is the maximum rate of 
information transfer: 

C = max/(£,77) (B7) 

where the maximization is performed over the collection of pairs £ and 77 that describe the channel. The fundamental 
Shannon theorem says that a source signal can be transmitted over a channel, if the source entropy is less than the 
channel capacity. 

Suppose finally that we have an input £ and we wish that the output 77 contains only some partial information about 
£, saying that some fidelity criterion is fulfilled. We consider all channels consisting of the fixed source £ and different 
outputs 77. The least rate of information transfer needed to specify £ in this way is source entropy with respect to the 
fidelity criterion: 

R = min M, 77) P ( fixed (B8) 
n 

where the minimization is performed over all 77 which satisfy the criterion with £. 
In practice many fidelity criteria can be written 

G(£,77)<6, (B9) 

where G is some suitable function and e is a measure of the fidelity. In this case the source entropy is naturally 
considered as a function of e, and we have the Kolmogorov e-entropy 

H(£, e) = min /(£, rj), where G(£, rf) < e and Pt fixed. (BIO) 

v 
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2. Continuous random variables 



Suppose we have a ra ndom variable with a continuous distribution, that is P^(dx) — p^{x)dx with continuous 
density (x). Then (B3) is infinite. 

As stressed by Kolmogorov, the mutual information, ( |B5|) , the capacity, (B7), and, in particular, the rate of 
information production with respect to a fidelity criterion, (p38,B10), are all well-defined also in the continuous case. 
In other words, although a real number observed with infinite accuracy contains an infinite amount of information, a 
real number observed with finite accuracy contains only a finite amount of information. 

We can see this in a simple heuristic way as follows: if we introduce a discretization of the space X into boxes with 
diameter S, we have a new random variable that we can call £5 with entropy 



,/ln(i 



Pt{x) hip^(x)dx + 0(6), 



(Bll) 



where d is the dimension of X . 

Similarly, from the conditional variable t;\r) we have a new random variable £g\r) with equivocation 

(H(ts\v))v = dln - J Pv(y) yj Pt\v( x >y) ln Pz\v( x >y) dx ) dy + o(s). 



Rearranging terms as in (B6), the mutual information between £g and r\ is thus 



PiA x ^v) 
pd x )Pv(y) 



dxdy + (D{5). 



(B12) 



(B13) 



As the discretization tends to zero the mutual information tends to a finite value, provided that the joint probability 
P^ v (dx, dy) is not singular with respect to the product P^(dx)P v (dy), a result due to Yaglom and Gel'fand [ pi) . 

We can give a meaning to the most random continuous distribution with respect some given constraints by max- 
imizing the entropy of a discretized continuous variable, as in (Bll), subject to these constraints, and then letting 
the discretization tend to zero. The most random distribution with given first and second moments is thus, not 
surprisingly, a Gaussian distribution with the same first and second moments. The entropy of a discretization of a 
d-dimensional real Gaussian random variable £ with correlation matrix C is 



fl"(&) = dln 



In 



Y / (2 7 re) d detC , + C'((5). 



(B14) 



All these results on the Gaussian distributions are due to Shannon [11 C 



3. The e-entropy result 



The spaces X and Y are now identical. We consider first a one-dimensional Gaussian random variable, and then 
a finite-dimensional variable and show that the Kolmogorov formula holds in these cases. In the next subsection, 
we then observe that the Fourier components of Gaussian processes and Gaussian random fields are independent 
Gaussian random variables, and so, taking the the formal infinite-dimensional limit, we find the desired result. 

The fidelity criterion will be the mean square deviation: 

((£-t?) 2 )<£ 2 (B15) 

Let us first estimate the answer by dimensional arguments. If the finite-dimensional normal variable £ is decomposed 
in its principal components, a fluctuation in direction i will be of typical size Oi. To estimate the fluctuation in 
direction i with an accuracy e we need about ln(<7i/e) bits. The answer should therefore be of the form 

ff(£,e)=A£m(^)+B. (B16) 

<Ti>£ 

The result of the analysis will be that A is one and B is zero. 

Let £ be a Gaussian one-dimensional random variable with variance a 2 . The random variable S}rj is defined on the 
support of P v . We suppose it to have mean m(y) and variance a(y). We have the obvious inequality 
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t a{y)P n {dy) < (£ 2 ), (B17) 

and the equality 

' a(y)P r ,(dy)={^-rj) 2 ). (B18) 



On the other hand, the equivocation, ( B12| ), with fixed outcome y of 77 and given mean m(y) and variance a(y) is 



maximized if the variable is Gaussian with variance a(y). Therefore, the mutual information between £ and r\ is 
bounded from below by 

/«,»?) > j P n {dy). (B19) 

We can therefore pose a constrained minimization problem over the auxiliary positive function a(y). Strictly speaking, 
H(£ , e) is only bounded from below by the minimization, but, as we will see, the solution can be realized in terms of 
variables £ and -q with desired properties. 

JT(£, e) = min / \ In ( P v (dy), (B20) 



a (y)J 2 \a(y) / 

e 2 > f a(y)P n (dy), (B21) 



a 2 > / a{y)P n (dy). (B22) 



Either one or the other of the two constraints will be satisfied at the minimum in ( B20| ). A variation with Lagrange 



multipliers gives that a(y) must be constant, either equal to e or to a . We therefore have 



H(£, e) = max In - , . (B23) 



The derivation also shows clearly that the solution can be realized as follows: if a is less than e, then (\r\ is Gaussian 
with mean zero and variance <r 2 , and r\ a delta function with support at the origin. If, on the other hand, a is greater 
than e, then 5\r\ is Gaussian with mean m(y) equal to y and variance e 2 , and r\ is Gaussian with mean zero and 
variance a 2 — e 2 . By the semigroup property of Gaussian kernels follows in both cases that £ is Gaussian with mean 
zero and variance a 2 . 

The generalization to higher-dimensional case is as follows. The variable £ is Gaussian with second moments CV,-, 
with principal components in a diagonal basis (er 2 , . . . ,o^). At a point yi,...,yd in the support of P v we consider 
the variable with first moments . . . , yd) and second moments (2/1, ■ ■ ■ , yd)- We have, in analogy with the 

onc-dimcnsional case, 



aaivi, ...,y d ) P v (dy) < (C 2 ) = af, (B24) 



and 



J ^a ll ( 2 / 1 ,..., 2/d )P t) (dy)-((e-7 7 ) 2 ) <e 2 . (B25) 

The mutual information between £ and 77 is bounded by 

I(C,V) > J\ (In det dj - In det a l3 ) P n (dy) (B26) 



Some of the inequalit ies ( B24) may hold as equalities in the solution, some not. It does however follow from the 



diagonal structure of ( B24 ) and ( B25 ) and the relation 



5oi 

<5 In det a = Tr — , (B27) 
a 
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that when (B26) is minimized, ctij(y) must be constant in y and diagonal in i and j. Let the diagonal elements of a 
be Df. We then have the following simpler discrete constrained minimization problem: 



ff(£,e)=min£> vz) 

i 

e 2 >E^> 



of > Df. 



(B28) 
(B29) 
(B30) 



The minimum can be found by starting with all D^s very small and increasing them proportionally to the gradient, 
that is, at the same rate. When on e of the ZVs hits the constraint (B30) we keep it constant from thereon and 
increase the others. The constraint (B29) will eventually be fulfilled when the ZVs that still change are equal to 9, 
and there we stop. The solution is thus implicitly given in terms of 9, which can be interpreted as a cut-off threshold 
for modes where normal fluctuations are small: 



E 



max 



In 



(5). 



E min K 2 ^ 2 ] 



(B31) 



The solution can be realized by taking r\ a random variable with independent components, diagonal in the same basis 
as £, and letting the i'th component of £ only depend on the i'th component of r\. The variables rji and £i\r)i are then 
constructed as in the one-dimensional case, with the only difference that the parameter 9 substitutes for the error e. 



4. Gaussian random fields 



Let us now consider a scalar Gaussian random field in Z?-dimcnsional space. A particular example (D — 1) is 
Gaussian processes. We begin by the approximation that the field is periodic with period L in all directions. F ourie r 
components of Gaussian random fields are independently distributed Gaussian random variables. Therefore (B31) 
can be rewritten, using the volume element Afc, equal to (2n/L) D : 



k 



In 



Ak. 



min \<r 2 



k ■ 



9 2 ] Ak. 



(B32) 
(B33) 



In the limit when L tends to infinity the right hand sides turn into integrals, and the left hand sides to entropy and 
mean-square distance per unit volume. Therefore we have 



h Y0lume (Z,e) 



1, [E(k)\ n 



/ min [E(k),9 2 ] d D k. 



d D k, 



(B34) 
(B35) 



which, for D equal to one, is the Kolmogorov result for the e-entropy per unit time of a Gaussian random process. 

We note that 9 is still a cut-off to eliminate modes such that the energy of the fluctuations in these modes is less 
than 9 2 . In fields with a power-law spectrum, such as turbulence, 9 can simply be substituted for a wave-number 
cut-off K. 



5. Other distributions and other fidelity criteria 

The mean square fidelity criterion is convenient for analytical computations, but it is not the only one possible. 
Shannon lists also the maximum distance, another choice would be the mean absolute distance. 

More interestingly, there may be cases where the most appropriate fidelity criteria is not simply proportional 
to the distance in the mean or in maximum value. If the signal £ is a Levy-process it will have a non-negligible 
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probability of changing by a large amount over a short interval of time 144] . In many applications one would then 
like an approximating signal rj to capture well the large jumps, but one would be less interested in a very precise 
approximation when f changes comparatively little ]45| , A reasonable fidelity criterion would then be that there is 
only a small probability that the distance between £ and rj is large. 

As far as we know no investigations have been performed of the entropy of such sources with respect to such fidelity 
criteria. Of course, for a Levy process a mean square error function is not possible, since the second moment is 
infinite. It would be possible to use mean absolute error, as was done implicitly by Gaspard and Wang in Jl3| , but 
the relevance of such computation would have to be motivated in each case by the application at hand. 



APPENDIX C: SPACETIME PROCESSES 



In this appendix we want to discuss and compare the finite-size Lyapunov exponent and the e-entropy for a turbulent 
flow and for the shell model. We assume that the flow and the shell model are both Gaussian as in appendix [B], and 
we assume that the spectrum follows the Kolmogorov theory, as described in section |J Non Gaussian effects and 
intermittency correction to the spectrum are not taken into account. 

At a given error threshold 8, we have seen that the finite-size Lyapunov exponent scales as X(8) ~ 8~ 2 , and this holds 
as well for the shell model as for 3D turbulence. Let us now consider the e-entropy, and first give a simple dimensional 



estimate. It will then be seen that the expressions (B34) (B35) just reproduce this result. The dimensional estimate 



starts from the time scales in the Kolmogorov theory, r(fc) ~ k 3 . The distance between two fields that only differ 
in wave numbers greater than k is 8 2 ~ k~$ ( that is, it does not depend on the dimensionality of space, but only of 
the form of the spectrum E(k) ~ . On the other hand, the number of degrees of freedom at wave numbers less 
than or equal to k is proportional to kP . The system must be observed at a rate r(/c) _1 to capture all the motion in 
wave numbers less than k. The amount of information per unit time and space needed to describe the system up to 
an error tolerance 8 is thus k D r(k)^ 1 . Translating this into a functional dependence on 8 we have 

fcspace.timep) „ 6 -2-sd (C1) 
which, for D = 3, leads to /jSpace.time^) _ J -u < As far 

as we can see there is an inconsistency between this result, 
also derived in ]l3j , and the result stated in ]l2j , which we believe is a misprint resulting from using an argument that 
really applies to the finite-size Lyapunov exponent and not the e-entropy. The very different scaling laws h ~ <5~ n 
and X(8) ~ 8~ 2 is also a further motivation why introducing the quantity finite-size Lyapunov exponent. 

This straight-forward dimensional analysis can easily be generalized to a generic stationary Gaussian processes with 
spectral density (see section 6.2.6 of 

*(k,w) (C2) 

The function F is supposed to vanish for argument much larger and much smaller than unity, and its integral is on 
the order of unity. The energy spectrum is therefore 

E(k) ~ k D ~ x J $(k, uj)duo ~ k z - y+D - 1 . (C3) 

We will here check that the previous dimensional estimates can also be obtained from the Kolmogorov formulae 



(B34) and (B35), that we write in this case as follows: 

/ 1 \ D + 1 

^space,time^ _ / 1 j 



1 f$(k,u) 
— In 1 



d D kdto, (C4) 



J min [$(k,w),6» 2 ] d D kduj. (C5) 



$ 2 = ( 7^ 

V2tt, 

The first observation is that the integral over u> in ( |C5| ) gives a contribution of order k z ~ y if k~ y is less than 9 2 , 
but otherwise a contribution of order k z 9 2 . The error is thus determined by a cut-off wave number K such that 

S 2 ~ k*-v+d K- y - 9 2 (C6) 



The second observation is that the integral over u in ( p4[ ) gives a vanishing contribution is k v is less than 6 2 , that 
is, for wave numbers larger than the cut-off K . For smaller wave numbers the integration over u) gives a contribution 
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of the order of k z , up to a logarithmic correction that we don't take into account. The e-entropy with cut-off wave 
number K is thus 



^space.timem ^ k_ z+d K~ v 



Combining (C5) and (C4) we find 



h(S) ^ 5 nD+z)/(z+D-y) 



(C7) 



(C8) 



which is equation (6.14) of p3| . By inserting the values D = 3, y = 13/3 and z = 2/3 of the Kolmogorov theory we 
reproduce the result (CI). 
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5v 

FIG. 1. Error doubling times (diamond) compared with shell turn-over times (plus). Number N of simulated shells is 27, 
and Reynolds number Re = v~ = 10 9 , ko = 0.05 and / = (1 + i) X 0.005. The equations were integrated with a slaved-frog 
scheme J2f|^(|, with constant time-step 2 • 10 -6 . The initial perturbation was randomly uniform over all shells in the inertial 
range, with amplitude less than 10 -6 . The perturbed and unperturbed configurations were integrated until the error reach the 
first threshold 5 at 10~ 4 . The error growth rate parameter r is 2 1//2 . The number of error doubling experiments was 400. The 
dashed line has slope —2. 
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In (Sv) 

FIG. 2. ln(l/T r (8v)) versus In (5v) for different Reynolds numbers Re = v~ x . Parameters as in Fig|l|, except that the 
times-step has been adjusted to the changing viscosity. The different symbols refer to: N = 24 and v — 10~ 8 (diamond); 
N = 27 and v = I0~ 9 (plus); N = 32 and v = I0" 10 (square); N = 35 and v = KT 11 (cross). The straight line has slope -2. 
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In [ Sv / Re" 1/4 ] 

FIG. 3. In [{l/T r (Sv))/Re 1/2 ] versus In [(^/Re" 1/4 ] at different Reynolds numbers Re = v~ x . Parameters as in Fig.@, 
except that the times-step has been adjusted to the changing viscosity. The different symbols refer to: N = 24 and v — 1CF 8 
(diamond); N = 27 and v = I0 -9 (plus); N = 32 and v = ICT 10 (square); N = 35 and v = I0 -11 (cross). The straight line has 
slope —2. 
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FIG. 4. Multiscaling data collapse [see eq. (S3)]. Parameters as in Fig.ji], except that the times-step has been adjusted to the 
changing viscosity. The different symbols refer to: N = 24 and v = 10 -8 (diamond); N — 27 and v — 10 -9 (plus); N — 32 and 
v = 10~ 10 (square); = 35 and v = 10" 11 (cross). The straight line has slope —2. The fitting parameters are R — 6 x 10 6 , 
V a = 5x 10~ 2 , and Re = 
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FIG. 5. Scale dependent Lyapunov exponent X(Sv) for two-dimensional shell model with N = 24 shells, fco = 0.05, v = 10 
and / = (1 + i) x 0.005. Note the linear scales on the ordinate axis. 
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FIG. 6. (1/T r (5i>))/Re 1/2 as a function of <5«/Re~ 1/4 for different values of the Reynolds number Re = v' 1 for the EDQNM 
approximation. The different symbols refer to: v = ICP 8 (diamond), v — ICP 9 (plus) and v — 10~ 10 (square). The line gives 
the Lorenz results —2. The other parameters are N = 32, fco = 0.05, e = 1 and a = 0.06. 




5v 

FIG. 7. (1/T r (5v)) (diamond) as a function of Sv for the EDQNM approximation of GOY model with N = 32, k = 0.05, 
e = 1 and v = I0~ 10 . The plus are the inverse of the eddy turn-over times t~ 1 (Sv) — k n E}J 2 versus Sv = (2NE n ) 1 ^ 2 . The 
straight line has slope —2. 
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FIG. 8. Scale dependent Lyapunov exponent X(S) for the map (|AS|). The line has slope 
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